Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IO-1332][internal] E2E Test harness #632

Merged
merged 5 commits into from
Jul 13, 2023
Merged

[IO-1332][internal] E2E Test harness #632

merged 5 commits into from
Jul 13, 2023

Conversation

owencjones
Copy link
Contributor

Problem

We lack a simple way to run darwin commands and take output for assertions

Solution

Write a simple wrapping function that runs darwin commands, and returns exit code, and STDOUT and STDERR streams outputs

Changelog

Added run_cli_command function, which wraps CLI commands
Added tests for run_cli_command

@linear
Copy link

linear bot commented Jul 13, 2023

IO-1332 Create CLI Testing Harness

Pytest has poor support/no support for test CLI commands, so we need a testing harness for the CLI, that is capable of:

  • Running a shell command
  • Handling the output and returning:
    • Output to STDOUT, STDERR as applicable
    • The shell return code, a signed 8-bit integer in POSIX systems
  • Making assertions based on this

E.g.

def test_it_creates_a_dataset():
    stdout, stderr, output_code = run_cli_command("darwin dataset create test-dataset")

    assert stdout.contains("test-dataset created")
    assert output_code == 0
    assert not stderr

This repo that I started work on some time ago may be of help, but is maybe a bit more than we need, but the file https://github.com/owencjones/cli_test/blob/main/cli_test/types.py contains a lot of what we need. It creates an expectation class, which we could use, but the functional approach above is just as good.

@owencjones owencjones changed the title Io 1332 [IO-1332][internal] E2E Test harness Jul 13, 2023

@mock.patch("e2e_tests.helpers.run")
def test_does_not_pass_working_directory_to_run_cli_command(mock_subprocess_run: mock.Mock) -> None:
mock_subprocess_run.reset_mock()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this shouldn't be necessary, patch resets automatically after exiting a decorator

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true, originally, I was wrapping several tests in a with mock.patch but removed it because it's very messy. I'll correct in the next ticket because it's in the same area

@owencjones owencjones merged commit eee5ca7 into master Jul 13, 2023
@Nathanjp91 Nathanjp91 deleted the io-1332 branch November 8, 2023 10:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants